Using Particle Swarm Optimization to Pre-Train Artificial Neural Networks: Selecting Initial Training Weights for Feed-Forward Back-Propagation Neural Networks

Performance 1 of supervised training of Artificial Neural Networks (ANNs) depends on several factors, including neural network architecture, number of neurons in hidden layers, the neurons' activation functions, and selection of initial network parameters (connection weights). Trial and error is commonly used to select the network parameters and the initial connection weights. Such practice can be prohibitive with large, complex solution spaces and large datasets [4]. This study shows that by applying a relatively light-weight, simple, and quick Particle Swarm Optimization (PSO) pre-training phase, the average network training, validation, and testing performance is improved significantly.